Fast and reliable head pose estimation algorithm is the basis of many high-level face analysis tasks. In order to solve the problem of existing algorithms such as illumination changes,occlusions and large pose variations,a new deep learning framework named HPENet was proposed. Firstly,with the point cloud data used as input,the feature points were extracted from the point cloud structure by using the farthest point sampling algorithm. With feature points as centers,points within spheres with several radiuses were grouped for the further feature description. Then,the multi-layer perceptron and the maximum pooling layer were used to implement the feature extraction of the point cloud,and the predicted head pose was output by the extracted features through the fully connected layer. To verify the effectiveness of HPENet,experiments were carried out on the Biwi Kinect Head Pose dataset. Experimental results show that the errors on angles of pitch,roll and yaw produced by HPENet are 2. 3,1. 5 and 2. 4 degree respectively,and the average time cost of HPENet is 8 ms per frame. Compared with other excellent algorithms,the proposed method has a better performance in terms of both accuracy and computational complexity.